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ABSTRACT 



Due to legislative mandate, the Ohio Department of Education 
(ODE) was required to develop a system (the Education Management Information 
System) that would increase the amount of information available to 
state-level policy makers and the public. Some recommendations for improving 
the function of EMIS are offered in this report. The text provides an 
overview of EMIS's establishment and past efforts to reform the system. 
Detailed information, such as the EMIS data elements, reliance on a computer 
network, funding history, and the process of submitting EMIS data, is 
offered. Concerns that emerged from a study of the system centered around the 
low priority afforded EMIS by the ODE, the inadequacy of the system, a lack 
of communication, and the cool interest in EMIS at the district level. Some 
of the recommendations for improving EMIS include heightening the accuracy of 
its data, clarifying expectations for EMIS, using more individual student 
data, and converting EMIS to a relational database. It is argued that for 
EMIS to function better, the ODE must lead all efforts to improve the system. 
Six appendices feature a selected bibliography, fluctuation analysis of EMIS 
data elements, the EMIS data elements, and other information. (ROM) 
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Summary 

Improving Ohio’s Education Management Information System (EMIS) 



Background 



The information submitted by 
districts through the EMIS is 
more accurate than when paper 
forms were used. 


In 1989, the General Assembly required the Ohio Department of 
Education (ODE) to increase the amount of information available about 
Ohio schools to state-level policy makers and the public. ODE was 
given less than two years and limited staff and financial resources to 
develop a computerized database, the Education Management 
Information System (EMIS). 

ODE designed the EMIS as an electronic version of much of the same 
information (student, staff, and financial) that was previously collected 
on paper forms. The ability to electronically cross check the figures 
provided by school districts resulted in more consistent and accurate 
information than what was submitted prior to the creation of the EMIS. 


ODE’s focus on a reporting 
system combined with the 
inability to collect individual 
student data reduced the 
capability of the EMIS to 
respond to questions for policy 
analysis. 


By designing a system that provides only pre-defined reports, ODE fell 
short of satisfying the needs of policy makers who also wanted a system 
that could respond to new and changing policy questions. As a result, 
state-level users continue to ask questions that cannot be answered by 
the EMIS and have difficulty interpreting and using the data that are 
available. The General Assembly farther reduced the capability of the 
system to answer policy questions by not allowing individual student 
data to be collected at the state level. 

By statute, individual student data must be aggregated before it is 
submitted to ODE by districts. ODE relies upon the existing Ohio 
Education Computer Network (OECN) and its 24 self-governing data 
acquisition sites to act as “collection points” for districts submitting their 
EMIS data to ODE. 

Since 1989, over $79 million state dollars have been allocated to the 
EMIS and an additional $144 million have been provided directly to the 
OECN as a state subsidy for EMIS and other services. 

Current Climate 

Within the last year, policy makers passed two significant pieces of 
legislation focusing on districts’ performance and fiscal accountability, 
S.B. 55 and H.B. 412. As a result, information on district and school 
performance will be published in a new “report card.” If these 
accountability objectives are to be fully realized, the EMIS data must be 
as accurate and complete as possible. 


0 
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The new accountability 
standards (S.B. 55 & H.B. 412) 
increase the urgency for a 
quality system. 



While Ohio is viewed by other states and national researchers as a 
forerunner in collecting elementary and secondary education data, the 
increasing reliance on the EMIS to inform the new accountability 
standards increases the urgency for a quality system. 



The current system for 
submitting and editing data is 
cumbersome, prone to errors, 
and burdensome to districts. 



The low priority given to the 
EMIS within ODE has resulted 
in an overall lack of 
collaboration among divisions, 
absence of a basic 
understanding, and poor 
technical support. 



Improved communication 
between ODE and districts 
would result in more accurate 

data. 



LOEO Conclusions 

Each school district must submit 202 data elements to ODE to satisfy the 
statutory requirements of the EMIS. Over 5,000 educators in Ohio need 
to understand how the EMIS functions in order to provide accurate and 
complete data to the state. The current system for submitting and 
analyzing data is cumbersome, prone to errors, and not well-understood 
by district and state-level users, even those within ODE. Furthermore, 
the process of aggregating the student data before submitting it to ODE 
makes correcting errors all the more difficult. 

In order for the existing system to function better at the state, data 
acquisition site, and district levels, the Ohio Department of Education 
must lead all efforts to improve the EMIS. ODE should begin by 
making the EMIS a higher priority. The lack of authority given to the 
Information Management Services division, the lack of collaboration 
among ODE divisions, and the lack of basic understanding of how the 
system works indicate the low level of priority ODE gives the EMIS. 

There is no group of ODE employees with knowledge of educational 
programs and how these programs are translated into computer coding 
for the EMIS. The message that the EMIS is not a priority at the state 
level trickles down to districts. 

While it is unrealistic to expect all 202 data elements to be 100% 
accurate, steps can be taken by ODE to improve the overall accuracy of 
the data. ODE could begin by reducing the acceptable margin of error 
for specific data elements and developing procedures for districts to 
verify the data entered into the EMIS against source documents. 

Many school districts view the EMIS as a low priority and a burdensome 
state mandate. Districts with this view typically employ part time EMIS 
coordinators, have an inadequate process for collecting and entering data 
into the system, and have the most difficulty submitting accurate EMIS 
data on time. Very few districts have one person who understands and is 
responsible for all three types of data - student, staff, and financial. 

In addition, very few district administrators see the value of the EMIS to 
better inform educational practice. In fact, most district administrators 
do not use the system for local purposes at all. 

While it is clear that school districts need help in changing their 
perceptions and mindset about using the EMIS, ODE could help change 
district perceptions significantly by better communicating how the EMIS 
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An evaluation of the quality of 
services provided by data 
acquisition sites is needed to 
improve technical assistance to 
districts. 



In order to meet the 
expectations of the 
accountability standards, 
improvements must be made to 
the technical support, 
communication, accuracy, and 
accountability of the EMIS. 



operates and how the data are used at the state level. ODE could also 
simplify the EMIS-related documents provided to districts, including the 
data definition and procedure manual ( EMIS Guidelines) and improve its 
technical assistance to data acquisition sites and school districts. 

ODE could lead the effort to improve technical assistance by evaluating 
the quality of services provided by data acquisition sites. One-third of 
the data acquisition sites provide quality services to school districts, 
while the quality of services provided by the rest is questionable. As the 
licensing agent for data acquisition sites, ODE has the authority to 
revoke their operating licenses, but has elected not to exercise this 
authority. In addition, state subsidy could be withheld from poorly 
performing data acquisition sites. 



Recommendations 

To fulfill the accountability requirements of S.B. 55 and H.B. 412, 
LOEO believes that the following recommendations should be 
implemented by ODE and school districts to improve the overall 
operation of the current system. 

Ohio Department of Education 

Despite the urgency under which the system was designed, ODE has 
never given the EMIS the necessary level of priority. The lack of 
coordination and collaboration among the various divisions has negative 
effects on the quality of the EMIS data, usability of the system, and the 
technical support provided to districts. 

LOEO recommends that the Ohio Department of Education: 

S Increase the priority of the EMIS within the Department beginning 
with a comprehensive evaluation of its internal infrastructure, 
equipment, and expertise. The results of the evaluation should be 
used to develop a strategic plan for the Department that places the 
EMIS at the center of its information needs; 

^ Improve technical support and communication to districts and data 
acquisition sites, including the adoption of a “customer service” 
mindset; 

S Improve the accuracy of the EMIS data by reducing the acceptable 
margin of error for specific data elements, develop model data 
verification procedures, use a fluctuation analysis to check the year- 
to-year change in value of more data elements, and pursue an 
external audit of the data contained in the EMIS; and 
^ Improve the accountability of the EMIS by formally evaluating the 
quality of services provided by data acquisition sites. 



School districts 



To ensure that the EMIS satisfies the reporting and accountability 
requirements of S.B. 55 and H.B. 412, LOEO recommends that school 
districts: 

'S Elevate the status and authority of the EMIS coordinators within the 
districts by assigning them responsibility for the coordination of all 
student, staff, and financial data. 



Policy Options 

If the General Assembly is interested in pursuing a system that allows 
for quick and sophisticated analysis of data that is more accurate, LOEO 
offers two additional policy options for consideration. Depending on its 
expectations for the EMIS, the General Assembly could implement one 
or both of the following policy options. 



Eliminating the October 
reporting period would improve 
the accuracy and usability of the 

EMIS. 



Option 1: Reduce Reporting Periods to Improve Accuracy 

Because state funding for schools is based on student attendance during 
the first full week in October, districts are required to submit all 59 
student data elements at that time. The data reported in October 
represents a frozen point in time, referred to as the “snapshot.” To 
reflect the changes that take place throughout the course of the school 
year, districts are required to update all 59 student data elements again in 
June. 



The problems associated with correcting the snapshot file account for 
many of the inaccuracies in the EMIS as well as its burden on schools. 
By eliminating the October snapshot from the EMIS, districts and ODE 
could focus on reporting accurate and timely data for the June reporting 
period. Basing school funding on the previous year’s data would help 
improve the accuracy and usability of the system. 

LOEO recommends: 

'S The General Assembly base state funding on school districts' 
previous year’s data. 
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Option 2: Alter EMIS’ Structural Design to Improve 
Performance 



The lack of individual student 
data at the state level and the 
absence of a relational database 
reduce the capabilities of the 
EMIS to answer complex policy 
questions. 



Although legislators’ expected a “flexible database” when the system 
was authorized in 1989, two essential features have always been absent 
from the system to completely fulfill this expectation: 1) ODE did not 
create a relational database; and 2) the General Assembly prohibited 
individual student data from being collected at the state level. The 
EMIS can be transformed into a relational database and technology 
enables individual student data to be collected at the state level without 
students being personally identified. 

Collecting individual, non-personally identifiable student data at the 
state level helps school districts in correcting their EMIS data. At the 
state level, individual student data make the EMIS a more flexible 
database for answering complex questions on program impact and the 
longitudinal effects of policy decisions. 

A relational database would improve the efficiency, accuracy, usability, 
and flexibility of the EMIS. Also, it would allow more in-depth 
analyses and more sophisticated questions to be answered by the EMIS. 
A relational database is necessary if ODE collects individual student 
data. 



If the General Assembly wants to improve the performance and 
capabilities of the EMIS by creating a flexible management information 
system that allows quick access to detailed data at the state level, 

LOEO recommends: 

S The General Assembly allow ODE to collect individual student data 
that is not personally identifiable. 

^ ODE convert the EMIS from a flat file structure into a relational 
database. 
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CHAPTER I 
INTRODUCTION 



Ohio's new accountability standards, outlined in S.B. 55 and H.B. 412 of the 122 nd General Assembly, 
make the Education Management Information System (EMIS) the most important source of information 
for evaluating public elementary and secondary schools. 



Background 

The Education Management Information 
System (EMIS) was created in 1989 by Am. 
Sub. Senate Bill 140 of the 118 th General 
Assembly. The Ohio Department of Education 
(ODE) was required to develop a system which 
increased the amount of information available to 
state-level policy makers and the public. The 
initial legislation reflected the following 
purposes for the EMIS: 

■ obtain uniform data for various input and 
output measures (e.g., number of teachers, 
test scores); 

■ compare schools and districts across the 
state; 

■ tie operating costs to output measures for 
efficiency ratings; 

■ provide numerical data by school, grade 
level, and subject area to be used in the 
identification of excellent and deficient 
schools and districts; and 

■ establish a flexible database for answering 
complex questions regarding schools and 
districts. 

To respond to this mandate, ODE 
developed a computerized database consisting of 
detailed information on students, staff, and 
finances. Much of the information ODE had 
been collecting on paper forms was transferred 
to the EMIS for electronic submission. To 
comply with the requirements of S.B. 140, a 
very complex computer system had to be 
developed in less than two years with limited 
staff and financial resources. The haste with 
which the EMIS was designed and implemented 
affects its operation to this day. 



Current climate 

Within the last year, policy makers have 
increased the level of accountability for 
elementary and secondary education in Ohio. In 
July and August, 1997, the 122 nd General 
Assembly passed two significant pieces of 
legislation focusing on districts’ performance 
and fiscal accountability, S.B. 55 and H.B. 412. 
Measures such as dropout rates, attendance 
rates, and the percentages of students passing the 
4 th , 6 th , 9 th , and 12 th grade proficiency tests will 
be used to judge the academic performance of 
schools and districts. 

As a reporting mechanism, S.B. 55 
requires an annual “report card” to be made 
available to parents and community members. 
H.B. 412 gives the Auditor of State authority to 
conduct performance audits of districts who are 
in a state of fiscal watch or fiscal emergency. 

If the objectives of S.B. 55 and H.B. 412 
are to be fully realized, the EMIS data must be 
as accurate and complete as possible. When 
LOEO examined compliance with the EMIS 
reporting requirements, we found that ODE’s 
definition of compliance does not consider the 
accuracy of the data to the degree needed. 
Districts could be in compliance with the EMIS 
reporting requirements and submit inaccurate or 
missing data elements. 

Recently, policy makers and district 
officials have raised a number of concerns 
regarding the accuracy of the EMIS data. In the 
most recent budget (Am. Sub. H.B. 215), the 
General Assembly addressed these concerns by 
fiscally sanctioning districts that do not submit 
accurate EMIS data by the specified deadlines. 
In addition, district superintendents and 
treasurers risk having their professional licenses 
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suspended or revoked for “willfully reporting 
erroneous, inaccurate, or incomplete EMIS 
data.” 

It is important to realize, however, that 
information about Ohio schools has become 
much more accurate since the creation of the 
EMIS in 1989. The replacement of hundreds of 
paper forms with electronic submissions has 
resulted in more consistency across records 
within schools and districts. The ability to 
electronically cross check figures produces more 
accurate data than when paper forms were 
completed by different district personnel and 
never compared. Furthermore, the data within 
the EMIS has become much more accurate since 
its first year of operation. 

Ohio is viewed by other states and 
national researchers as a forerunner in collecting 
elementary and secondary education data. Ohio 
was recently selected by the National Center for 
Education Statistics as its only site for studying 
how school districts use education data for 
decision making. 

However, the increasing reliance on the 
EMIS to inform the new accountability 
standards necessitates an even greater urgency to 
evaluate the quality of the system. The 
expectations for what the EMIS can and should 
be able to do continue to grow, but the design of 
the system remains unchanged. These design 
issues, as well as the difficulties that surfaced 
while the system was being implemented, should 
be addressed immediately , since the EMIS is 
being relied upon for the new accountability 
initiatives. 

Focus of this report 

As testimony to legislative interest in 
the development and quality of the EMIS, this is 
the fourth time the Legislative Office of 
Education Oversight (LOEO) has been asked to 
study the system since 1990. Previous reports 
have focused on the status of its implementation 
(1990), an overall assessment of its 
implementation and operations (1993), and 
whether school districts were complying with its 
reporting requirements (1997). 



This fourth LOEO report identifies 
major concerns with the EMIS and proposes 
significant strategies for improving it. 

Methods 

The following five steps summarize 
LOEO’s methods for conducting this study. 

1. Reviewed over 70 documents related to 
computer management information systems. 
(See Appendix A for a selected 
bibliography.) 

2. Conducted site visits to 13 school districts 
and four data acquisition sites; five 
additional data acquisitions sites were 
interviewed by telephone. 

3. Interviewed staff from the Ohio Department 
of Administrative Services, State Auditor’s 
Office, and various divisions within the 
Ohio Department of Education. We also 
interviewed state legislators and legislative 
staff, a database design specialist, a data 
specialist from the National Center for 
Education Statistics, and officials from 12 
states with education management 
information systems. 

4. Convened two focus groups comprised of 
school district employees who work on the 
EMIS to discuss their proposed solutions to 
difficulties with the system. 

5. Conducted a fluctuation analysis on various 
data elements to identify possible 
inaccuracies by tracking unusually large 
changes from year-to-year. (See Appendix 
B for further details on the fluctuation 
analysis.) 

Report organization 

The next chapter describes the structure 
of the EMIS and its process for handling data. 
Chapter III provides LOEO’s findings about the 
overall concerns with the EMIS, and Chapter IV 
describes ways to improve its accuracy and 
usability. LOEO’s conclusions and 

recommendations are in the final chapter. 



CHAPTER II 

STRUCTURE AND PROCESS 



The structure of the EMIS and the process for handling data are complex . Understanding the structure 
and process is necessary to follow our discussion about the concerns with the current system. 



EMIS data elements 

Through the EMIS, 202 data elements 
on students , staff, and finances are collected and 
electronically transmitted to ODE. Exhibit 1 



provides the number of each type of data 
element and some examples of the range of 
information required. A complete listing of the 
data elements and their respective reporting 
dates is in Appendix C. 



Exhibit 1 

Data Elements Collected in the EMIS 



Type of Data 


Number of Data 
Elements 


Examples 


Student 


59 


date admitted; number of days in attendance; gender; in a 
vocational program; in an athletic program; courses taken; 
proficiency test results 


Staff 


49* 


certification status; pay rate; courses taught; funding source 


Financial 


94 


current fund balance; inside millage; number of certificated 
employees; total assessed valuation 


TOTAL 


202 





* includes 10 data elements pertinent to the district as a whole, such as number of days in session 



Reliance on the Ohio Education 
Computer Network (OECN) 

ODE relies on an existing computer 
network, the Ohio Education Computer Network 
(OECN), as the vehicle for electronic 
submission of the data. The OECN was created 
in 1979 by the 113 th General Assembly to 
provide cost-effective accounting services to 
school districts, including the electronic 
transmission of their financial data to ODE. 

The OECN consists of three levels of 
organization: school buildings and districts; 24 
regionally located “data acquisition sites” 
(formerly A-sites), and the Ohio Department of 
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Education. (See Appendix D for a picture of the 
OECN structure.) 

The 24 self-governing data acquisition 
sites are crucial to this network because they 
share computational power and specialized 
software with school districts. Since all city, 
exempted village, local, joint vocational school 
districts, and education service centers are 
required to report their data through the OECN, 
over 726 districts use the network for EMIS 
purposes. 

By statute, student data must be 
aggregated before it is submitted to ODE. 
Because current law prohibits the collection of 
individual student data at the state level, the data 
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acquisition sites serve as “collection points” for 
districts submitting their EMIS data to ODE. As 
a result, the technical assistance and other 
services provided by data acquisition sites are 
essential to school districts successfully carrying 
out their EMIS responsibilities. 

Funding the EMIS and the OECN 

Since 1989 over $79 million state 
dollars have been allocated to the EMIS. An 
additional $144 million have been provided 
directly to the OECN data acquisition sites as a 
state subsidy. EMIS is only one of the many 
services provided to school districts by the 



OECN. The state invests a significant amount of 
funding in the OECN for EMIS and other 
services provided by the data acquisition sites to 
school districts. 

In several budgets since 1989, the 
General Assembly has substantially increased its 
funding to the EMIS for various improvements. 
Although state funding for the OECN had 
remained relatively constant from fiscal years 
1989 to 1994, the General Assembly increased 
its funding in fiscal year 1995 by 80% to support 
the network’s role in the SchoolNet initiatives. 
Exhibit 2 lists the state funding for both the 
EMIS and OECN since 1989. 



Exhibit 2 

State Investment in the EMIS and OECN since 1989 



House 

Bill 

111 


General 

Assembly 

118 th 


Biennium 

1989-1991 


EMIS Funding 
(Line Item GRF-446) 

$7,i 044, 696 


OECN Funding 
(Line Item GRF-426) 

$20,258,040 


Total Investment 

$ 27,302,736 


298 


119 th 


1991-1993 


$13,500,000 


$20,723,124 


$34,223,124 


152 


120 th 


1993-1995 


$18,000,000 


$21,323,136 


$39,323,136 


117 


121 st 


1995-1997 


$18,428,411 


$38,240,380 


$56,668,791 


215 & 650 


122 nd 


1997-1999 


$22,360,331 


$43,926,937 


$ 66,287,268 


TOTAL 






$79,333,438 


$144,471,617 


$223,805,055 



Process for submitting EMIS data 

To carry out their EMIS responsibilities, 
school districts start by entering information 
about students, staff, and finances into their own 
local databases. Often these local databases are 
run on the computers located at the regional data 
acquisition site. Once entered, the data elements 
required for EMIS reporting are “extracted” 
from the local database and stored in a separate 
file. 

LOEO found that the staff and financial 
data are typically entered in the district’s central 



administrative office, usually by the treasurer’s 
secretary. However, the student data are initially 
entered at the building level. School building 
secretaries are responsible for entering the 59 
data elements required for each student by the 
EMIS, plus any additional elements they need 
locally. 

Managing the student data is the most 
burdensome part of the EMIS for school 
districts. Many of the concerns about the EMIS 
stem from the process by which the student data 
must be handled. Exhibit 3 portrays the various 
steps in the process of handling the student data. 
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Exhibit 3 

Process for Handling Student Data 
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Step 1: 

Student data are entered into the 
districts' local "active" database 



EMIS 

Coordinator 



Step 2: 

Data are "extracted" from 
districts' "active" databases to 
create three snapshot data files. 




School 

Buildings 



Step 3: 

Districts run edit checks. 
(Errors corrected if found.) 



Step 4: 

Student data are aggregated either by the 
district or the data acquistition site. 



Individual 

Student 

Data 

Snapshot 




Staff Data 
Snapshot 




Financial j 
Data 
Snapshot 



Step 5: 

Districts run simulation reports 
(Errors corrected if found.) 




Data 

Acquisition 

Sites 




IF YES 




Step 6: 

24 data acquisition sites collect the 
aggregated student data along with staff 
and financial data and forward to ODE. 



Step 7: 

ODE runs edit checks and 
forwards error reports to districts. 



IF NO 
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Data must be corrected in the snapshot file 
by EMIS coordinator 



Snapshot file. An important aspect of 
the process is that data are extracted from the 
local databases several times a year and reported 
to ODE. The most important extraction for the 
student data occurs in October, because under 
current law school funding is based on the 
number of students enrolled in the district in the 
first full week of October. This extracted data 
represents a frozen point in time and is referred 
to as a “snapshot.” 

After building secretaries enter 
information on each student, the data are sent to 
the “EMIS coordinator” who extracts the 59 
required EMIS data elements to create the 
student “snapshot” file. This snapshot file 
remains separate from the district’s “active” 
database. The active database continues to 
change as the daily operations of the district 
proceed, such as having students enter and leave 
the district, while the snapshot file remains 
frozen. 

For at least three weeks after the 
snapshot is taken, districts run numerous edit 
checks on the snapshot file to ensure a certain 
level of accuracy. An edit check will test 
whether an entry meets some predetermined 
standard. For example, the “gender” data 
element will only accept the values “M” or “F” 
and any other entry in this field is flagged as an 
error. 

Once the district believes the student 
snapshot is as accurate as possible, the district 
aggregates the student data. At this point, 
districts run simulation reports on their 
aggregated data which attempt to mimic the type 
of error checking procedures done by ODE. 

Finding errors. The data acquisition 
sites forward the aggregated student snapshot 
files, as well as those for staff and finances, to 
ODE. Once received, ODE runs a series of 
more refined edit checks on the data and 
generates “error reports” for the districts. For 
example, an ODE error report might compare 
the number of vocational units for which the 
district is approved against the number of 



vocational education students recorded in EMIS. 
If the number of students is “short” of the 
number approved, the district must check for 
possible errors because its unit funding for 
vocational education is in jeopardy. 

The lack of individual student data at the 
state level becomes a problem for districts when 
they have to correct the errors identified in the 
ODE error reports. Because the ODE edit 
checks are run on aggregated student data, ODE 
cannot tell the district where the error is, only 
that there is an error somewhere in the file. 
Therefore, districts must “guess” where the 
errors may have occurred and start checking 
student records contained in the snapshot file. 
As districts reported to LOEO, it takes them 
longer to find the error than it does to actually 
correct it. For districts with large student 
populations, this problem is particularly 
burdensome. 

As noted, the snapshot file reflects what 
was taking place in the district during the first 
full week of October. This may differ 
considerably from what is actually taking place 
when districts start correcting errors in the file. 
Because the district’s “active” database is 
updated almost daily to reflect current activities, 
it is not helpful for finding an error in the 
snapshot file. For example, new students who 
enrolled in vocational education after the 
snapshot was taken cannot be confused with 
those who were enrolled the first week of 
October. Instead, school staff have to find the 
students who were participating in vocational 
education during the first week of October but 
whose student records are mistakenly coded as 
not participating. 

The more days that elapse from when 
the snapshot was taken, the more difficult it 
becomes to correct the snapshot file, especially 
given that 59 data elements are required for each 
student. It is quite easy to mistakenly code one 
or more of these elements in October. This is 
especially true for urban districts, whose large 
and mobile student populations do not begin to 
stabilize until November. 




6 



15 



Districts must be careful to correct 
mistakes contained in both the snapshot and live 
files, otherwise the same mistake will occur 
again when the snapshot is updated. 

Adding to the troublesome nature of 
correcting the October snapshot file is the fact 
that it will have to be updated at the end of the 
school year anyway. Recognizing the number of 
activities occurring during a given school year 
(e.g., proficiency testing, student transfers 
between various programs of study, etc.), 
districts are required to reconcile their October 
data with an updated snapshot in June. The data 
reported in June are also used by ODE to adjust 
state funding and to generate the new S.B. 55 
report cards. 



By eliminating the October snapshot 
from the EMIS, districts and ODE could focus 
their energies on reporting accurate and timely 
data for the entire school year. Because the data 
are based on an entire school year and not a 
single week in October, they are more 
meaningful to districts and state-level users. 



Summary 

The current method for basing school 
funding on data from the first full week of 
October significantly impacts the accuracy of 
the EMIS data. Basing school funding on the 
previous year’s data would help improve the 
accuracy and usability of the system. 



CHAPTER III 

CONCERNS WITH THE EMIS 



As the leader of elementary and secondary education , 

ODE should lead all efforts to maintain and improve the EMIS. 



One of the primary responsibilities of 
ODE is to provide information about public 
elementary and secondary education to districts, 
legislators, and the general public. As the 
primary source of education data for the state, 
the EMIS plays an integral part in ODE 
fulfilling this responsibility. However, after 
seven years of implementation, districts and 
state-level users still do not understand the 
purpose of the EMIS or its capabilities. This is 
due in large part to the design of the system. 

ODE designed the EMIS as a reporting 
system to primarily fulfill its federal and state 
reporting requirements. By focusing only on the 
reporting rather than analysis requirements of 
S.B. 140, ODE fell short of satisfying the needs 
of policy makers who wanted a system that 
could be queried to answer their questions. 

The General Assembly further reduced 
the analysis capabilities by not allowing 
individual data to be collected at the state level. 
As a result, state-level users, such as the General 
Assembly and its staff, continue to ask questions 
that cannot be answered by the EMIS and have 
difficulty interpreting and using the data that are 
available. 

Low priority within ODE 

Despite the urgency under which the 
system was designed in 1989, ODE has never 
given the EMIS the necessary level of priority. 
As a department, ODE is comprised of over a 
dozen divisions such as School Finance, Special 
Education, and Information Management 
Services (IMS). Each division is responsible for 
the management and oversight of their 
respective areas or programs and often their own 
separate databases. Yet, LOEO found there to 
be very little coordination or collaboration 
among the divisions concerning the EMIS. 







Coordination among ODE divisions. 

The lack of coordination and collaboration 
among the various divisions has negative effects 
on the quality of the EMIS data, the usability of 
the system, and the technical support provided to 
districts. For example, the accuracy and 
timeliness of the EMIS data submitted by 
districts is directly affected by the late arrival of 
key support documents, such as the EMIS 
Guidelines (a data definition and procedure 
manual) provided by ODE. 

Although responsible for operating the 
EMIS, the IMS division has no authority to 
require other divisions within ODE to “share” 
the information that is necessary to make the 
EMIS operational. Throughout the course of 
this study, LOEO learned of “turf” battles 
among the various divisions that ensued from 
the IMS division trying to exert leadership in 
coordinating the EMIS and other Department 
information. 

Because the IMS division lacks the 
authority to amend the EMIS Guidelines without 
the approval of other divisions, the document is 
delayed and districts do not receive it far enough 
in advance of the October reporting deadline to 
make the necessary changes in their data. This 
directly impacts the quality and accuracy of the 
data districts submit to ODE. 

In addition, LOEO found no “core 
body” of ODE employees with knowledge of 
educational programs and an understanding of 
how EMIS operates. For example, area 
coordinators in the Division of School Finance 
may be helpful in answering financial questions, 
but they do not understand the technical aspects 
for entering and reporting EMIS data. Similarly, 
the computer programmers in the IMS division 
lack a depth of understanding for how the 
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educational programs must be translated into the 
EMIS codes. 

School districts and data acquisition 
sites call divisions within ODE for technical 
assistance and receive different answers to the 
same questions. The director from one data 
acquisition site provided examples of 

representatives from ODE sharing information 
with each other for the first time during an EMIS 
workshop. This lack of collaboration and 

coordination among the various divisions creates 
the perception that the EMIS is a low priority for 
ODE. 

Staff shortages. The understaffed IMS 
division is another example of the low priority 
of the EMIS. Reductions in the IMS staff 
significantly impact the quality and type of 
services provided to data acquisition sites, 
districts, and other users of the EMIS data. 
Since 1990, the number of IMS staff has 
declined from 46 to 34. While some of these 
reductions may be due to attrition and 
departmental budget cuts, there are currently 
four key positions that remain unfilled, including 
the Executive Director. 

In part, the staff shortages within the 
IMS division are a result of “market forces.” 
Similar to other government agencies, ODE has 
had difficulty recruiting and retaining 
individuals with the necessary knowledge and 
skills because of higher paying jobs in the 
private sector. Three of the fifteen programmers 
have recently left the IMS division for private 
sector opportunities. 

Summary 

The low priority given to the EMIS 
within ODE, the lack of collaboration and 
coordination among its various divisions, and 
the lack of understanding of the EMIS by ODE 
staff impact the quality of EMIS data submitted 
by districts. These practices convey to districts 
that the EMIS is not a priority for ODE, 
therefore, it is not a priority for districts. 



Usability of the system 

Districts, data acquisition sites, and 
state-level users consistently reported to LOEO 
the difficulties they encountered using the 
EMIS. Most districts have problems using the 
system because it is not “user-friendly.” For 
example, school district officials described not 
being able to locally generate reports because of 
the numerous programming codes needed to do 
so. As a result, they must rely on the services 
provided by their data acquisition sites for 
accessing and navigating their way through the 
system. 

State-level users also expressed 
frustration over the usability of the system, as 
well as the condition of the data available on the 
ODE web site. Frequent users of the EMIS 
report to LOEO that the data must be 
significantly “cleaned up” before it can be used 
in their analyses. In fact, this task requires so 
much work that one state agency uses the data 
sets of another ODE division (not IMS) who has 
already corrected the data. 

EMIS software. While ODE developed 
a common software for districts to use when 
extracting the staff and financial data, no such 
software exists for the student data. When 
EMIS was initially developed, legislators urged 
ODE to allow districts to use their local vendors 
and thereby allow as much “local control” as 
possible. As a result, multiple commercial 
student software packages were adopted and 
used to report student data. These packages are 
not compatible with each other nor with the 
state-developed staff and financial packages. It 
is also not possible for ODE to provide technical 
support for the various packages, which 
increases districts’ reliance on data acquisition 
sites. 

In recognizing the many problems 
resulting from the lack of a common student 
software as well as the “year 2000” issue, the 
General Assembly authorized (through Am. Sub. 
H.B. 215) ODE “to procure or develop a 
common EMIS software.” However, ODE has 
chosen to focus only on a common student 
software rather than a common EMIS software. 
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While LOEO believes that this is a step in the 
right direction, it does not address the need to 
integrate the student, staff, and financial data. 
An integrated software package would reduce 
the manual repetition of data entry by school 
districts and would reinforce the relationships 
between student, staff, and financial data. 

To date, ODE has neither “procured” 
nor “developed” a common software package. 
Due to problems with the selected bidder, ODE 
cancelled the first Request for Proposal for a 
software package, and is currently pursuing 
alternatives. 

Technical support. Since ODE does 
not have enough staff to provide adequate EMIS 
assistance to school districts, it trains data 
acquisition site staff who in turn train district 
personnel. However, not all of the necessary 
information or understanding about the EMIS is 
reaching the districts through this approach. As 
a result, the inadequate training provided by 
ODE to data acquisition site staff negatively 
impacts the technical support provided to school 
districts. 

While the EMIS training of data 
acquisition site staff by ODE has failed, the lack 
of understanding regarding educational 
programs remains a problem. Similar to the 
IMS division of ODE, most data acquisition site 
personnel are “computer techies” who do not 
understand the interrelationships between the 
technical aspects of the EMIS and education 
programs. As a result, they are unable to 
transfer such knowledge to school district 
personnel. 

The IMS division of ODE responds to 
“frequently asked questions” about the EMIS 
through its mainframe computer system. 
According to data acquisition site and district 
officials, this is an effective method to get 
answers to questions that have already been 
addressed. However, the IMS division is far less 
effective in providing support services through 
other mediums. 

One data acquisition site official 
provided LOEO with documentation of 
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questions posed to ODE over an eight-month 
period. ODE failed to respond to eleven out of 
the 27 questions. The official noted, “getting 
answers to your questions is especially crucial 
for upcoming reporting deadlines. They’re even 
more critical now that there are severe financial 
penalties to the districts for not having their data 
submitted and accurate.” 



Summary 

ODE’s design of the EMIS for reporting 
its federal and state data has resulted in a system 
that is not user-friendly and does not adequately 
fulfill the needs of districts and state-level users 
of the data. 

ODE’s communication 

District officials repeatedly conveyed to 
LOEO that they do not understand how the 
EMIS data submitted to the state are used in 
calculations, ODE reports, or in policy making 
decisions. For the most part, this confusion and 
lack of understanding is a result of ineffective 
communication by ODE. 

Currently, there are few written 
materials available that clearly and concisely 
explain what EMIS is and what it is capable of 
doing. The only available document is the EMIS 
Guidelines , a procedural manual that defines 
how each data element is to be coded by school 
district personnel. For example, the staff data 
element “degree type” defines seven possible 
coding options for certified staff: 0 - non- 
degree, 1 - associate, 2 - bachelors, 3 - masters, 
4 - education specialist, 5 - doctorate, and 6 - 
other. 

The EMIS Guidelines is more than 500 
pages in length and very difficult to understand. 
This is particularly troubling given the critical 
role the EMIS Guidelines play in coding and 
interpreting EMIS data, primarily for districts. 

EMIS Guidelines. Districts, data 
acquisition sites, and state-level users all 
expressed negative opinions regarding the clarity 
and usability of the EMIS Guidelines. They 
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wished for a “guidebook” to the EMIS that is 
non-technical and written for the “average” 
person. LOEO’s review of the literature and 
interviews with audit agencies emphasize the 
necessity of establishing and communicating 
clear definitions of data elements. The lack of 
clear data definitions and procedures allow 
districts to interpret similar situations differently 
for coding purposes. 

An example of inaccurate coding based 
on an unclear data definition: the staff data 

element “authorized experience years” denotes 
the total years of authorized teaching experience 
for a district’s certified staff. ODE’s definition 
of what counts as authorized teaching 
experience spans two pages and includes 14 
criteria that are very difficult to understand. 
Because of the lack of clarity, it is likely that 
school secretaries are coding teachers with 
similar work experience in different manners. 
School officials told LOEO that they do not trust 
using the EMIS data to compare districts 
because of coding differences based on unclear 
data definitions. 

Districts interviewed by LOEO 
expressed that their greatest difficulties in 
coding came from the “unique situations” for 
out-of-district, special education, and vocational 
education students. Even though such students 
represent the smallest percentage of their 
populations, they require the greatest amount of 
time to code into EMIS. 

Even when ODE’s data definitions are 
adequately explained in the EMIS Guidelines , 
some school districts purposely miscode certain 
data elements which they philosophically 
believe the state should not be collecting. Such 
data elements typically contain very personal 
information about the student’s economic status 
(e.g., disadvantagement code) or the student’s 
reason for withdrawal from school. One school 
district official told LOEO that it does not report 
“pregnancy” as the reason for withdrawal in any 
of its female student records. Instead, this 
district labels the student’s withdrawal reason as 
“other.” 



Inconsistencies in coding decisions can 
invalidate any comparisons that are made when 
using the EMIS data. State-level users assume 
that any data they use from EMIS to be 100% 
accurate, when in reality it is not. For example, 
any state-level user trying to analyze the 
differences in staff experience between various 
types of school districts needs to assume a 
certain level of error for the data element 
“authorized experience years.” 

Because the EMIS Guidelines do not 
adequately meet the information needs of district 
and state-level users, there is considerable 
confusion over the meaning of certain data as 
well as ODE’s calculations. One of the most 
commonly cited examples is the graduation rate. 

Graduation rate. The graduation rate 
is one of the most important yet least understood 
state calculations among policy makers and 
school district officials. Districts expressed 
considerable confusion over the difference in the 
graduation rate reported by ODE from what they 
calculated locally. When asked why these 
differences occur, ODE explained that the lack 
of individual student data at the state-level 
requires a series of formulas that “statistically 
control” for such factors as student dropouts and 
mobility. ODE explained that these “statistical 
formulas” are the source of the confusion. 

However, LOEO found that the inability 
to understand the graduation rate calculation is 
due in part to the formulas used by ODE, in 
addition to the inadequate explanations of these 
formulas provided in the EMIS District Profile 
Reference Manual or those provided by the IMS 
division. 

Throughout the course of this study, 
LOEO made several attempts to acquire a 
comprehensible explanation of the graduation 
rate from ODE, but each time we were directed 
to a reference manual. Currently, the only 
explanation of the graduation rate appears in the 
EMIS District Profile Reference Manual which 
is no easier to use or understand than the EMIS 
Guidelines. LOEO still does not have an 
understandable explanation of how the 
graduation rate is calculated. 
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As long as the data definitions, state 
calculations, and the use of data are poorly 
communicated, districts will continue to distrust 
or misrepresent the EMIS data. As one large 
urban district stated, “we have a ‘large city 
attitude.’ We generate our own data and tell the 
media not to trust the EMIS data.” 

Over 5,000 staff in schools, data 
acquisition sites, and ODE need to clearly 
understand the system in order to submit 
accurate and timely data. When districts and 
other users do not understand and cannot 
replicate calculations for key data elements, they 
question the accuracy of the data. 

Summary 

None of ODE’s documents suitably 
explain the definitions for each of the data 
elements, the relationships between various data 
elements, or how they are used in state 
calculations. This reinforces districts’ 
perception that the EMIS data are inaccurate, 
which adds to their reluctance to use or trust the 
data. 

Low priority within districts 

The low priority that ODE gives the 
EMIS “trickles down” to the district level. Most 
EMIS coordinators have difficulty completing 
all of the required duties, mostly because they 
are part time. EMIS coordinators have other 
duties in addition to their EMIS responsibilities. 
It could also be due to their lack of authority 
over other district staff working on the EMIS, 
such as secretaries and treasurers. 

LOEO was surprised to learn that most 
districts do not have a truly “coordinated” 
process for handling all three types of EMIS 
data. Typically, EMIS coordinators are only 
responsible for the student data while treasurers 
manage the financial and staff data. By dividing 
the responsibilities for the EMIS across several 
individuals within the district, there is no single 
person with a complete understanding of the 
data or how the system works. This often results 
in distrust of the data’s accuracy. 



LOEO found that districts who submit 
the most accurate and timely information have 
EMIS coordinators who understand the inter- 
relationships between data elements, the impact 
of their coding decisions on funding, and the 
process for submitting data. In addition, these 
districts typically have full time EMIS 
coordinators with the authority to ensure that all 
EMIS obligations are fulfilled by all district 
staff. 



Summary 

Districts who submit the most accurate 
and timely data recognize the importance of the 
EMIS and give it the appropriate level of 
priority in their district. Typically, these 
districts assign EMIS responsibilities to a high- 
level administrator who oversees all three types 
of data (student, staff, financial) and has the 
authority to ensure that all aspects of EMIS are 
carried out at the building and district level. 

Use of data for local decision making 

The majority of district administrators 
interviewed by LOEO do not recognize the 
value in using EMIS data. In fact, most believe 
that the only data of merit are those in their local 
databases that can be used for the daily 
operation of a district. Most district 
administrators contend that the “historical” and 
“summary” nature of the EMIS snapshot data 
make them less useful for local decision making. 
Most of these administrators characterize the 
EMIS data as “unrepresentative of what is really 
taking place in the district.” However, the few 
district administrators who do use EMIS data 
share a common characteristic — an 
understanding of how data better inform their 
practice. 

While many district administrators do 
not use the EMIS data because they do not trust 
its accuracy, others reportedly spend their 
energy strategizing how to react to the public 
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display of their EMIS data rather than learning 
how it could be used to improve the educational 
practices within the district. 

Within the last few months, ODE has 
been working with professional associations 
such as Buckeye Association of School 
Administrators and Ohio School Boards 
Association to train superintendents, treasurers, 
and school board members on how to use EMIS 
data. While ODE’s efforts are a step in the right 
direction, more is required. School districts and 
ODE differ in the types of information they use. 
To date, ODE has not gained a thorough 



understanding of school districts’ needs or the 
impediments that currently prevent them from 
using EMIS data. 

Summary 

Districts invest a significant amount of 
financial and human resources into EMIS but 
most believe that they gain very little from it. If 
districts believed that they were getting some 
benefit from the EMIS, they would harbor less 
resentment, care more about the data (which 
would improve accuracy), and eventually use the 
data to inform local decisions. 



CHAPTER IV 

IMPROVING THE ACCURACY AND USABILITY OF THE EMIS 



The accuracy and usability of the EMIS could be improved by adding editing procedures designed 
specifically to test the accuracy of the data and by altering the structure of the system to 
better match the current expectations for its use. 



Currently, school districts are solely 
responsible for ensuring the accuracy of EMIS 
data. In the most recent state budget (H.B. 215), 
the General Assembly created a new 
accountability system that gives ODE the 
authority to fiscally sanction districts or suspend 
or revoke superintendents’ and treasurers’ 
licenses for submitting late or inaccurate data. 
This should shorten the length of time districts 
spend correcting EMIS data. 

However, the district officials LOEO 
interviewed believe it is the responsibility of 
both ODE and data acquisition sites to help 
districts meet the EMIS requirements; technical 
assistance from both is essential for school 
districts to meet these responsibilities. 

Even though data acquisition sites 
receive a significant amount of state funding, 
there are no standards to judge the quality of the 
services they provide to school districts. In 
LOEO’s 1996 report, Ohio SchoolNet 
Initiatives: The Role of the Ohio Education 

Computer Network , we examined the quality of 
services that data acquisition sites provided to 
school districts. We found that eight of the 24 
data acquisitions sites were “good,” eight were 
“adequate,” and eight were “poor.” LOEO 
recommended that all 24 data acquisition sites 
be formally evaluated and state subsidy be 
eliminated for those identified as providing poor 
services. 

There has been no formal evaluation of 
data acquisition sites even though ODE is 
required by State Board of Education standards 
to conduct such reviews. Even the Management 
Council of the Ohio Education Computer 
Network (MCOECN), the coordinating body of 
the network, voted in February 1997 to 
encourage ODE to evaluate the data acquisition 



sites according to state standards; however, no 
evaluations have taken place to date. 

Accuracy of EMIS data 

Other states with education management 
information systems have ongoing concerns 
about data accuracy. States such as Florida and 
Texas, whose systems were operating at least 
three years prior to Ohio’s, are still struggling to 
increase the accuracy of the data in their 
systems. These states are increasingly using 
their education management information 
systems as the primary data source for assessing 
the performance of public schools. 

LOEO contacted a number of state and 
national research organizations and states with 
education management information systems to 
find standards of accuracy for management 
information systems. LOEO found no agreed- 
upon “standards” for judging the accuracy of 
EMIS data. However, all of the people LOEO 
contacted agreed that it is unreasonable to expect 
all of the data to be 100% accurate. 

LOEO also found that an “acceptable” 
margin of error should differ for each data 
element. Information used as a basis for funding 
should have a smaller margin of error than 
information unrelated to funding. For example, 
Average Daily Membership (ADM), which is 
the basis of funding for Ohio public schools, 
should be more accurate than the number of 
students enrolled in extracurricular activities. 

ODE and districts told LOEO that the 
EMIS data elements related to school funding 
and those that are used frequently for other 
purposes are the most reliable. ODE’s error 
checking procedures focus mostly on data used 
for funding purposes. In addition, now that 
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staffing ratios and proficiency test results are on 
the report cards, more attention is being paid to 
the accuracy of this information. In fact, ODE 
warns EMIS users not to trust data elements that 
are unrelated to funding or infrequently used. 

According to an expert from the 
National Center for Education Statistics, the best 
approach to increasing the accuracy of 
information is “to provide a basis for developing 
measures to prevent errors.” Similarly, a 
General Accounting Office report found that it 
costs more to correct data after it is entered 
rather than before. 

Editing and correcting data. Ohio 
relies on computer edit checks for ensuring the 
accuracy of the EMIS data. There are basically 
two types of edit checks. One type determines if 
the data entered matches the range of acceptable 
values, such as “M” or “F” for the gender data 
element. A second type tests whether entered 
numbers lie outside a specified range. For 
example, if the number of days a student is 
reported to have attended school exceeds the 
number of possible days school is open in a 
given year, the number is flagged as an error. 

However, neither of these approaches 
verifies the initial accuracy of the data. In other 
words, the number of days recorded for the 
attendance of a particular student may be within 
the acceptable range of days that school is open, 
yet the number may not accurately reflect the 
days that student was actually present. 

Neither ODE nor school districts have 
controls in place that test the accuracy of the 
data. While some districts interviewed by 
LOEO compare the data they enter into the 
system with source documents (e.g., emergency 
forms, attendance sheets, etc.), most districts do 
nothing more than review the error reports 
provided by ODE. Verification procedures that 
compare source documents with the information 
entered into the system are what is needed to 
ultimately measure the accuracy of the data. 

Results of the fluctuation analysis. 

Most of ODE’s data verification procedures only 
compare data elements across reporting periods 



within the same year from October to June. 
Comparisons across years only occur for three 
data elements (ADM, staff count, and unit 
funding). Examining changes across years is 
another method of assessing the accuracy of 
information in the EMIS. 

To gain some insight into the level of 
accuracy of EMIS data, LOEO conducted a 
fluctuation analysis. This analysis identifies 
possible inaccuracies in data by tracking 
unusually large changes in data element values 
from year to year. Officials from the Texas 
Department of Education conducted a similar 
analysis by comparing two years of data and 
discovered hundreds of instances where a 
change of over 200% occurred. They 
subsequently confirmed that over three-fourths 
of these instances were actual errors in the data. 
Texas recommended that all of the data in the 
system be audited. 

Of the 202 data elements in EMIS, 
LOEO examined six elements for the two years 
ending in 1995 and 1996: Average Daily 

Membership (ADM); student attendance; staff 
attendance; student dropouts; student retention; 
and total handicapped students. 

LOEO looked for any changes from 
year to year that exceeded 50%, 100%, and the 
generous threshold of 200%. LOEO found some 
instances of a 200% change for three of the six 
data elements. These instances raise the 
possibility of an error in the data that warrants 
further examination; they do not automatically 
mean the data are incorrect. Appendix B 
provides more detail on the results of the 
fluctuation analysis for the 50% and 100% 
thresholds. 

LOEO believes it would be helpful for 
ODE to use a fluctuation analysis as another 
way to check for errors in more data elements. 
A 200% threshold is overly generous, however, 
for some types of data. As noted, the experts we 
contacted agreed that an acceptable margin of 
error should differ for each data element. For 
data that impact funding levels, the acceptable 
margin should be much smaller. 
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For example, during the 1997-1998 
school year, an audit by the Auditor of State 
found eight districts’ ADM in error by as much 
as five percent. According to the Auditor, a five 
percent error is too high for data that are used as 
a basis for funding, especially for large districts. 
A five percent error in ADM for Columbus 
Public Schools equals approximately 3,150 
students. Based on the foundation level funding 
amount outlined in Am. Sub. H.B. 650 ($3,851), 
this five percent error translates into about $12.1 
million for the 1998-1999 school year. 

Using our fluctuation analysis, we found 
six districts with changes in ADM between 10% 
and 20% for the years 1995-1996. To determine 
if these are actually errors, these data should be 
compared to the original source of the 
information within the school district. For 
example, ADM could be compared to 
enrollment forms. ODE could use this approach 
to directly assess the accuracy of the data in the 
EMIS. This effort would require a full-fledged 
audit. 

Expectations for the EMIS 

The expectations for the type of 
questions the EMIS should be able to answer 
have evolved while the structure of the system 
has remained the same. The nature of questions 
currently asked by policy makers reflect their 
desire to have a system that goes beyond basic 
reporting to one that allows for more 
sophisticated analyses. 

The increasing reliance on the EMIS 
data to evaluate schools and answer policy 
questions has resulted in requests for data and 
analyses that are not readily available through 
the EMIS. When these questions cannot be 
answered, policy makers become frustrated and 
lose confidence in the system. 

However, while legislators expected a 
“flexible database” in 1989 to answer detailed 
policy questions, two essential features were 
missing from the system that prevented this 
flexibility: 1) ODE did not create a relational 

database; and 2) the General Assembly 



prohibited individual student data from being 
collected at the state level. These features are 
still absent from the system today and impact the 
accuracy and usability of the EMIS. 

Individual student data at the state level 

The public and most policy makers are 
generally not aware that it is possible to collect 
individual student data at the state level without 
revealing the personal identity of students. This 
is accomplished by replacing a student name, 
social security number, date of birth, or other 
personally identifiable information with a unique 
identification code. Ten of the twelve states 
LOEO contacted collect individual student data 
at the state level. 

While some of these states use the social 
security number as the student identification 
code, most create a unique code unrelated to the 
social security number. In order for this unique 
identification code to be portable from one 
district to another, it is generated at the state 
level to prevent students from having duplicate 
identification codes. 

Benefits. There are a number of 
benefits to having individual student data. With 
unique identification codes, in combination with 
a common student software package, student 
files could be transferred between districts. 
Currently, only districts served by the same data 
acquisition site share files electronically. This is 
a particular concern for students receiving 
special education services. District officials 
reported having to wait up to three months 
before students’ individual education programs 
(IEP) are forwarded from other districts. 

Collecting individual student records at 
the state level would reduce the burden placed 
on school districts in identifying errors in 
aggregated data. In addition, more detailed 
analyses would be possible at the state level. 
Currently, state-level analysts and policy makers 
are limited to the results of pre-defined analyses 
made available by ODE, whose staff must try to 
predict questions that arise. 
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Legislators are increasingly interested in 
policy and evaluation questions that require 
individual student data. With aggregated student 



data, state-level analysts cannot answer 
questions such as: 



■ How many 4 th grade students who did not pass the 4 th Grade Proficiency Test after three attempts 
were promoted to the 5 th grade? 

■ Were students identified in 1 st , 2 nd , or 3 rd grade as “reading below grade level” provided with 
intervention services or summer school programs? If so, how many of these students passed the 
4 th Grade Proficiency Test? 

■ Do students who are “retained in grade” actually remain in school? In the long-term, does this 
policy benefit or harm students? 

■ What impact do class size and all-day kindergarten have on student performance? 

■ Do programs for at-risk students such as JOG, GRADS, OWE, or OWA increase graduation 
rates? 

■ Do programs such as Head Start and Public School Pre-school improve student achievement? 

■ What impact does Tech Prep have on student achievement? 



District officials expressed “mixed” 
reactions to collecting individual student data at 
the state level. Some districts were adamantly 
opposed to the idea, while others recognized the 
potential benefits. Based on parents’ opposition 
to providing social security numbers for their 
children, district officials anticipate that a large 
number of parents would be opposed to the 
state’s access to individual student data. To help 
alleviate this opposition, the district officials 
recommended clearly explaining to parents how 
individual student data could be collected 
without revealing the personal identity of 
students. 

Other states. The majority of states 
that LOEO interviewed cited collecting 
individual student data and its potential invasion 
of privacy as a major consideration in designing 
their state education management information 
systems. Still, ten of the twelve states that 
LOEO interviewed report having individual 
student data at the state level, including Florida 
and Texas, who are the forerunners in education 



management information systems. Appendix E 
provides more detail on other states’ education 
management information systems. 

Summary 

Collecting individual, non-personally 
identifiable student data at the state level would 
help school districts correct EMIS data more 
easily, which would also improve its accuracy. 
In addition, the EMIS would be able to quickly 
answer more detailed policy questions as well as 
provide data for longitudinal evaluations of 
state-funded programs. 

Converting the EMIS to a relational 
database 

Another limitation of using the EMIS 
for detailed analyses is its “flat file” structure, 
where all of the data are stored in separate, 
stand-alone files or tables. This type of file 
structure requires the same data to be stored in 
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multiple locations, creating a greater chance for 
inconsistencies and errors. 

There are no built-in relationships 
linking EMIS data elements stored in these 
separate tables; instead, separate computer 
programs must be written to establish the 
relationships. Other than the IMS staff at ODE, 
very few users of the system have the skill to 
create the necessary relationships among the 
data elements to perform detailed analyses. 

In contrast, a relational database stores 
data with pre-defined linkages or relationships. 
Information is stored at the most detailed level 
(e.g., individual student data) allowing users to 
easily “query” the system for detailed analyses. 
For example, student, staff, and financial data 
would be linked in the EMIS to allow a user to 
easily calculate the average per-student cost of 
teaching all industrial technology courses at a 
given high school. 

A relational database would be 
necessary if Ohio were to collect individual 
student data at the state level. Appendix F 
describes some common characteristics of 
relational databases. 

Other states. Ten of the 12 states 
LOEO contacted use relational database designs 
for their education management information 
systems because they believe it offers improved 



accuracy and maximum flexibility. (See 
Appendix E.) 

Converting the EMIS to a relational 
database requires proper planning and an 
adequate amount of time to design, field test, 
and implement the system. ODE has the 
hardware to support a relational database but it 
lacks the skilled personnel to design one. The 
EMS division is understaffed and consists mostly 
of programmers with little to no experience 
designing relational databases. 

A relational database would most likely 
need to be designed and installed by a private 
contractor. Given ODE’s budgetary and staffing 
restrictions, hiring someone with the necessary 
skills appears unlikely without additional 
funding. ODE has taken steps toward improving 
the usability of the EMIS by purchasing 
software that mimics a relational database. 
While this approach partially improves the 
usability of the system, it does not make the 
system fully relational. 

Summary 

A relational database would improve the 
efficiency, accuracy, and usability of the EMIS 
and allow for more detailed and in-depth 
analyses to address policy questions. 
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CHAPTER V 

CONCLUSIONS AND RECOMMENDATIONS 



This chapter provides a series of recommendations and two policy options. The recommendations are 
the minimum changes needed to make the Education Management Information System operate more 
effectively and fulfill the accountability requirements ofS.B. 55 and H.B. 412. In addition , members of 
the Ohio General Assembly could select one or both of the policy options 
based on what they expect the EMIS to be able to do. 



In order for the existing system to function better at the state, data acquisition site, and district 
level, the Ohio Department of Education (ODE) must lead all efforts to improve the EMIS. ODE should 
begin by making the EMIS a higher priority. The lack of authority given to the Information Management 
Services division, the lack of cooperation among ODE divisions, and the lack of basic understanding of 
how the system works by virtually all ODE staff indicate the low level of priority ODE gives the EMIS. 
Further, there is no group of ODE employees with knowledge of educational programs and how these 
programs are translated into computer coding for the EMIS. The message that the EMIS is not a priority 
at the state level trickles down to the district level. 

Many school districts view the EMIS as a low priority and a burdensome state mandate. Districts 
with this view typically employ part time EMIS coordinators, have an inadequate process for collecting 
and entering data into the system, and have the most difficulty submitting accurate EMIS data on time. 
Very few districts have one person who understands and is responsible for all three types of data - 
student, staff, and financial. 

In addition, very few district administrators see the value of the EMIS to better inform 
educational practice. In fact, most district administrators do not use the system for local purposes at all. 

While it is clear that school districts need help in changing their perceptions and mindset about 
using the EMIS, ODE could help change district perceptions significantly by better communicating how 
the EMIS operates and how the data are used at the state level. ODE could also simplify the EMIS 
Guidelines and improve its technical assistance to data acquisition sites and school districts. 

From an earlier study, LOEO found that one-third of the 24 data acquisition sites provide quality 
services to school districts, while the quality of services provided by the rest is questionable. As the 
licensing agent for data acquisition sites, ODE has the authority to revoke their operating licenses, but 
has elected not to exercise this authority. 



Recommendations 

LOEO believes that at a minimum the following changes should be made to improve how the 
current EMIS operates. 



Increase the priority of the EMIS within ODE . 

■ In the last two months, ODE hired an Information Technology Officer at the assistant superintendent 
level. This is a belated step in the right direction. This position should be responsible for overseeing 
all of the data management functions of ODE. The Information Technology Officer should make the 
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EMIS the principal data source for all decision making within ODE. Staff at all levels should be 
required to answer to the Information Technology Officer regarding all information needs. 

■ The Information Technology Officer should immediately conduct a comprehensive evaluation of 
ODE’s internal infrastructure for collecting and processing information for all purposes, 
particularly the information used for decision making and for disseminating to external audiences. 

■ The evaluation should include an assessment of the type of equipment used by each division, their 
sources of information, and the expertise of division staff in developing databases and other types 
of mechanisms for storing and processing information. 

■ The results of the evaluation should be used to develop a comprehensive information management 
plan for coordinating ODE’s internal information and identifying future needs. The EMIS should 
be at the center of this plan. 

■ Create a permanent information management group within ODE to coordinate all of the data 
management functions of the Department. The members of this group should possess the technical 
knowledge of how the EMIS operates and how educational program areas are translated or coded 
into the system. This group should be led by the new Information Technology Officer. All ODE 
employees should have at least a basic understanding of how the EMIS operates. 



To improve technical support and communication, ODE should: 

■ Develop and implement strategies to improve technical assistance to data acquisition sites and 
school districts. 

■ Explain the purpose of the EMIS to school districts and users about how data are used by state- 
level policy makers in an easy-to-read document other than the EMIS Guidelines. 

■ Shift from the primary purpose of fulfilling its own state and federal reporting requirements to a 
system that allows its users to analyze information to answer their questions. 

■ Begin viewing districts and other users of education data as its customers. 

■ Improve the EMIS software to increase the user-friendliness of the system. 

■ Improve the ODE web-site by including “raw” data, frequently used data sets, and better 
explanations of the layout and organization of the data. 
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■ Overhaul the EMIS Guidelines to make them more understandable for the typical district employee 
who may know very little about computers or how the EMIS operates, including: 

■ Clarify data definitions for consistent coding across school districts; 

■ Provide detailed, clear explanations of ODE calculations; 

■ Distribute the EMIS Guidelines well in advance of the next school year so school 
districts have time to make the required adjustments to their systems; 

■ Combine all of ODE’s reporting requirements into the EMIS Guidelines (e.g., special 
education and vocational education) rather than separate documents; 

■ Include page numbers in the EMIS Guidelines ; and 

■ Offer the EMIS Guidelines in both unabridged and summarized versions. 



To improve the accuracy of EMIS data , ODE should: 

■ Reduce the acceptable margin of error (tolerance ranges) for specific data elements. 

■ Develop model data verification procedures that districts could adopt to improve the accuracy of 
data. 

■ Use a fluctuation analysis to check the year-to-year change in value of more data elements. 

■ Pursue an audit of the data contained in the EMIS. 



To improve the accountability of the EMIS: 

■ Data acquisition sites should be formally evaluated by an independent entity against some state 
minimum standards, in addition to the local expectations of their member districts. Furthermore, 
ODE should revoke the administrative site licenses of data acquisition sites that provide poor 
service and should withdraw state subsidy from these sites. 



To ensure that the EMIS satisfies the reporting and accountability requirements of S.B, 55 
and H.B . 412 , LOEO recommends that school districts should: 

■ Elevate the status and authority of the EMIS coordinators within the districts by assigning them 
responsibility for the coordination of all student, staff, and financial data. 
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Policy Options 



Depending on its expectations for the EMIS, the General Assembly could implement one or both 
of the following policy options. 

Option 1: Reduce Reporting Periods to Improve Accuracy 

Because state funding for schools is based on student attendance during the first full week in 
October, districts are required to submit all 59 student data elements at that time. The data reported in 
October represents a frozen point in time, referred to as the “snapshot.” To reflect the changes that take 
place throughout the course of the school year, districts are required to update all 59 student data 
elements again in June. 

The problems associated with correcting the snapshot file account for many of the inaccuracies 
in the EMIS as well as its burden on schools. By eliminating the October snapshot from the EMIS, 
districts and ODE could focus on reporting accurate and timely data for the June reporting period. 
Basing school funding on the previous year’s data would help improve the accuracy and usability of the 
system. 



Districts will always increase or reduce in size, therefore, basing state funding on the previous 
school year could pose problems for districts with substantial changes in their student populations. 
However, a protocol for handling these exceptional cases could be developed. 



LOEO recommends that the General Assembly base state funding on school districts’ 
previous year’s data. 



Option 2: Alter EMIS’ Structural Design to Improve its Performance 

The EMIS was created in 1989 to provide information on student performance and district costs 
to state-level policy makers and the public. Although the expectations for what the EMIS should be able 
to do continue to increase, the structure of the system has not changed. The EMIS was designed 
primarily as a reporting system, but increasingly, state-level policy makers expect to use the system to 
provide detailed and sophisticated analyses which require combining the student, staff, and financial 
data. 



Although legislators expected a “flexible database” when the system was authorized in 1989, 
two essential features have always been absent from the system to completely fulfill this expectation: 1) 
ODE did not create a relational database; and 2) the General Assembly prohibited individual student data 
from being collected at the state level. The EMIS can be transformed into a relational database and 
technology enables individual student data to be collected at the state level without students being 
personally identified. 
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Collecting individual, non-personally identifiable student data at the state level helps the school 
districts in their correction of EMIS data. At the state level, individual student data make the EMIS a 
more flexible database for answering complex questions on program impact and the longitudinal effects 
of policy decisions. 

A relational database would improve the efficiency, accuracy, usability, and flexibility of the 
EMIS. Also, it would allow more in-depth analyses and more sophisticated questions to be answered by 
the EMIS. A relational database is necessary if ODE collects individual student data. 



If the General Assembly wants to improve the performance and capabilities of the EMIS by 
creating a flexible management information system that allows quick access to detailed data at the 
state level , LOEO recommends that: 

■ The General Assembly allow ODE to collect individual student data that is not personally 
identifiable. 

■ ODE convert the EMIS from a flat file structure into a relational database. 
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APPENDIX B 



FLUCTUATION ANALYSIS OF SIX EMIS DATA ELEMENTS 



LOEO conducted a fluctuation analysis to gain some insight into the level of accuracy of 
the EMIS data. This analysis identifies possible inaccuracies in data by tracking unusually large 
changes in data element values from year to year. Of the 202 data elements in the EMIS, LOEO 
examined six elements for the two years ending in 1995 and 1996. LOEO looked for any changes 
that exceeded three threshold ranges: 50%; 100%; and 200%. The exhibit below displays the 
results of the analysis. 

Number of Districts with Large Changes Between 1995 and 1996 
for Six EMIS Data Elements 



Data Elements 

Average Daily Membership (ADM)* 


50% 

(50-99%) 

1 


Magnitude of Changes 

100% 200% 
(100— 1 99%) (200% or more) 


Student Attendance** 


- 


— 


— 


Dropouts 


87 


34 


15 


Retention grades 1-8 


84 


70 


47 


Total Number of Handicapped Students 


2 


— 


1 


Staff Attendance 


1 


8 


“ 



* Six districts had changes in ADM between 10% and 20%. 

** Three districts had changes in attendance between 10% and 20%. 



These large changes raise the possibility of errors in the data that warrant further 
examination; they do not, however, automatically mean the data are incorrect. It would be 
helpful for ODE to use a fluctuation analysis as another method of checking for errors in EMIS 
data. As noted, for certain data elements, such as those related to funding, a much smaller margin 
of error should be applied. For example, for data that impact funding levels, the acceptable 
margin should be much smaller. A 200% threshold is overly generous. 
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APPENDIX C 



EMIS DATA ELEMENTS 

The following list includes 202 data elements collected on students, staff, and finances. 
The Auditor of State determined that all 94 financial data elements are required by law. Ninety- 
eight student and staff elements are required by state and/or federal law. The remaining 10 staff 
data elements are not specifically required by state or federal law, but are needed for verification 
of other data elements or for required calculations. 



Student Data 



Demographics 

1. Building IRN 

2. Date of birth 

3. Disability condition 

4. Disadvantagement 

5. District of residence 

6. Gender 

7. Grade level 

8. Grade level, next year 

9. Limited English proficiency 

10. Racial/ethnic category 

1 1 . Student ID 

12. Student name (optional) 

13. Student percent of time 

14. Student status 

Attendance 

15. Admission date 

16. Attendance (days) 

17. Attending/home district IRN 

18. Attending/home indicator 

19. Authorized absence (days) 

20. Award of Merit 

21. Corporal punishment 

22. Curriculum/completer status 

23. Date of withdrawal/ dropout/ truancy 

24. Diploma date 

25. Diploma type 

26. Expulsion 

27. Graduation credit units 

28. Majority of attendance IRN 

29. Non-attending reason 

30. Re-entry 

31. Suspension 

32. Unauthorized absence (days) 



0 




Programs 

33. Academic extra-curricular and intra- 
curricular programs 

34. Athletic extracurricular programs 

35. Child day-care 

36. DPIA 

37. Drivers education 

38. Early childhood service delivery options 

39. Educational options 

40. Enterprise programs 

41. Gifted educational programs 

42. Immigrant education program 

43. Open-enrollment 

44. Other regular programs 

45. Postsecondary enrollment options 

46. School related service programs 

47. Special education placement option 

48. Special education related services 

49. Special education unit 

50. Special programs 

51. Title I 

52. Vocational programs 

Subject 

53. Course code (for each course taken) 

54. Course grade 

55. Course status 

56. Local classroom code 

Competency-based Education 

57. Competency-based education results 

Proficiency Testing 

58. Proficiency testing results 

Summer School 

59. Student summer school 
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Staff Data 



Demographics 


23. Position start date 


1. * Absence (days) 


24. Position status 


2. Absence/long term illness 


25. Position type 


3. * Attendance (days) 


26. ^Scheduled work days 


4. Authorized experience years 


27. *Separation reason 


5. *Date of birth 

6. Degree type 


28. Type of appointment 


7. Employee name 


Course Master 


8. Gender 


29. Course code (subject) 


9. Racial/ethnic category 


30. Course level 


10. Semester hours 


3 1 . Course type 


11. Staff ID 


32. *Hours of operation for preschool 


12. Total experience years 


education/ Voc. Ed. 

33. Length of scheduled instruction 


Employment 


34. *Local classroom code 


13. Assignment area 


35. Location IRN (building) 


14. Certificate application 


36. Program provider IRN 


15. Contracted pay amount/rate 


37. Semester code 


16. Contracted pay type 


38. Teacher’s social security number (for a 


17. Extended service 


particular course) 


18. * Length of work day 


39. Vocational advisory committee 


19. Local contract code 


(optional) 


20. Position code 

21. Position FTE 

22. Position fund source 


District-wide Data 


1. Building IRN 


7. Lunchroom percentage 


2. Building square feet 


8. Number of days in session 


3. District IRN 


9. Transportation percentage 


4. Instructional plan 

5. *Kindergarten days in session 

6. *Last day of school 


10. Vocational education correlated classes 

Financial Data 



1 . Account description 

2. Amount 

3. Amount outstanding end of period 

4. Capacity of board 

5. Case number 

6. CFDA number 

7. Comments 

8. County name 

9. Court name 

10. Current cash encumbered 



13. Current payables 

14. Depository name 

15. Description of proceedings 

16. Description/purpose of issue 

17. District ERN 

18. Entity IRN 

19. Entity name 

20. Expense for proceeding for current fiscal 
year 

21. Federal contribution received 

22. Federal expenditure during current fiscal 
year 



11. Current encumbered 

12. Current fund balance 



* These data elements are not required by law, but needed fon verification or for 
required calculations. ^ ^ 



23. Fiscal year 

24. Fiscal year actual expenditure 

25. Fiscal year actual receipts 

26. Fiscal year ending 

27. Fiscal year revenue 

28. Fiscal year expenditures 

29. Fiscal year receipts 

30. Fiscal year receivable 

31. Fiscal year total appropriation 

32. Function 

33. Fund 

34. Fund class 

35. Grant title 

36. Inside millage 

37. Instructional level 

38. Interest 

39. Interest date 

40. Interest rate 

41. Issue date 

42. Job 

43. July 1 cash balance 

44. Line number 

45. Maturity date 

46. Millage - adjusted commercial/industrial 

47. Millage - adjusted residential/agricultural 

48. Millage - full assessed rate 

49. Millage - line number 

50. New issues during period 

51. Object 

52. Operational unit 

53. Outside millage 

54. Plaintiff/defendant name 

55. Plaintiff/defendant type 

56. Principal redemptions during period 

57. Principle 

58. Prior fiscal year encumbered 

59. Receipt 

60. Receiving fund 

61. Receiving special cost center 

62. Reporting period 

63. Schedule frequency 

64. Schedule number 

65. Schedule sequence 

66. Source 

67. Special cost center 

68. Statutory authority 

69. Subject 

70. Tax receipts - personal-general 

71. Tax receipts - personal-public utilities 

72. Tax receipts - real-commercial/industrial 



73. Tax receipts - real-minerals 

74. Tax receipts - real-public utilities 

75. Tax receipts - real-residential/agricultural 

76. Tax receipts - total tax receipts 

77. Tax valuation - personal-general 

78. Tax valuation - personal-public utilities 

79. Tax valuation - real-commercial/industrial 

80. Tax valuation - real-minerals 

81. Tax valuation - real-public utilities 

82. Tax valuation - real residential/agricultural 

83. Tax valuation - tax exempt 

84. Tax valuation - total assessed valuation 

85. Total 

86. Total assessed valuation 

87. Total average daily membership 

88. Total expense 

89. Total federal receipt group 

90. Total number of certificated employees 

91. Total number of non-certificated 
employees 

92. Transaction indicator 

93. USAS fund 

94. Year 

A tally of the data elements collected by EMIS: 



Student Data 


59 


Staff Data 


39 


District Data 


10 


Financial Data 


94 


Total Elements 


202 



EMIS Reporting Dates 



Reporting Period 


Groups of Data 

Student Staff Financial 


October 15 


X 


X 




October 30 






X 


December 20 


X* 






January 30 






X 


April 30 






X 


June 30 


X 


X 




July 30 






X 



* Special education enrollments only 
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APPENDIX D 



OHIO EDUCATION COMPUTER NETWORK (OECN) 
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Type of Checks or Audits 
(data accuracy) 

Still developing the mechanisms and procedures; 
currently considering verifying data at the local level. 


Audits of its student records. 


Edit and variance checks similar to Ohio. 


Compares electronic data with paper reports from school 
districts; considering variance checks in the near future. 


Edit and verification checks, along with audits of 
enrollment data. 


Unknown. 


Edit and verification checks; compare reported data 
against other departmental data (e.g., teacher 
certification). 


Unknown. 


Procedures being developed. 


Edit and verification procedures. 


School districts verify the data sent to the state; becomes 
permanent record. 


Verification checks and financial audits. 


Completely 

Implemented 

System 

No 


Yes 


Yes 


No 


Yes 


No 


Yes 


No 


No 


Yes 


Yes 


Yes 


Relational 

File 

Structure 

Yes 


Yes 


Yes 


Yes 


* 

* 
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<L> 


Yes 


No 


Yes 


No 


Yes 


Yes 


Yes 


Individual 


Student 

Data 

No 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


II Type of Data Collected 


Financial 

Yes 


Yes* 


Yes 

1 


No 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Staff 

Yes 


Yes* 


Yes 


No 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Student 

Yes 


Yes* 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


State 

Interviewed 

California 


Delaware 


Florida 


Iowa 


Maryland 


Massachusetts 


Minnesota 


Nevada 


South Dakota 


Texas 


West Virginia 


| Wyoming | 
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APPENDIX F 



CHARACTERISTICS OF RELATIONAL DATABASES 

As noted in the text of the report, 10 of the 12 states LOEO contacted use relational database 
designs for their education management information systems because they believe it offers improved 
accuracy and maximum flexibility over a flat file data base system. Some common characteristics of 
relational databases include: 

■ More efficient and accurate data collection. Because data resides in only one location rather 
than several records and tables, there are fewer chances for errors when entering or changing 
data. 

■ Greater flexibility and more sophisticated analysis. The increased flexibility allows for better, 
more sophisticated and detailed analyses of data, which in turn results in broader policy and 
evaluation questions being answered. 

■ Improved usability of the system. The usability of the system improves because the 
relationships between data elements are displayed to users. Most of today’s relational database 
software allows users to apply a common graphical querying tool, preventing the need for 
computer programming. 

■ Integration of all education data. A relational database would move the EMIS one step closer 
to becoming a “data warehouse” for educational information. The information from a data 
warehouse is more detailed and accurate, and provides the tools that enable the user to view, 
analyze, and report on data in ways that support decision making. Integrating ODE’s various 
databases (EMIS, teacher certification, special education, vocational education, assessment, etc.) 
into a data warehouse would offer more comprehensive and detailed educational information for 
policymakers and the general public. 
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State of Ohio 



Department of Education 



John M. Goff 



Ohio Departments Building, Room 81 (J, 65 South Front Street, Columbus 43215-4183 



Superintendent of Public Instruction 



May 27, 1998 



TO: 



Nancy Zajano, Director 



SUBJECT: 



FROM: 




The Department of Education is pleased to provide this response to the LOEO report Improving Ohio’s Education 
Management Information System ( EMIS ) . dated 13 May 1998. 

In general, the department finds itself in agreement with the findings of LOEO regarding the current state of the 
EMIS system. We appreciate having had the opportunity to provide input during the study and at the study’s closing 
conference. Although not all of our suggested input was used, we find the report accurate and timely. 

We also wish to acknowledge the validity of LOEO’s recommendations for actions to improve the EMIS system. 

For the most part, the department agrees with LOEO’s recommendations. In fact, as this response indicates, the 
department has already begun to implement many of the recommendations. Given that several recommendations 
require either a longer-term effort or a legislative change, our response indicates where we are actively engaged or 
where further planning or future actions are required. 

LOEO Recommendation: Increase the priority of EMIS within ODE* 

1 . Conduct a comprehensive evaluation of ODE’s internal infrastructure. 

ODE’s Information Technology Officer began a comprehensive review of technology in use by the department 
on 09 March. Preliminary findings were reported to the department’s senior management and to the State Board 
of Education on 12 May. Not surprisingly, these findings align with and support the majority of the 
recommendations of the LOEO regarding EMIS. 

2. Develop a comprehensive information management plan. 

As a result of the preliminary findings of a review of ODE’s technology use, ODE’s Information Technology 
Officer began a comprehensive information systems planning activity within the department to develop an 
information technology plan covering all departmental activities. The plan, when completed, will focus on 
short-term (9 to 12 months) and long-term (18 to 24 months) projects/activities and will involve all divisions of 
ODE. ODE expects this effort to be a continuing business process for the department and not a one-time event. 

3. Create a permanent information management group within ODE for EMIS. 

In May, ODE’s Information Technology Officer recommended the creation of an EMIS steering group to be 
staffed by members of the department’s division-level management. This recommendation was accepted by 
ODE senior management The EMIS steering group will be made up of ODE division directors and will be 
responsible for business ownership of the EMIS application. Business ownership includes, but is not limited to, 
extensive data planning and resource coordination among and between divisions. By establishing business 
ownership and staffing the steering group with ODE division-level management, EMIS will receive the attention 
and focus required of such a mission-critical application for the department at the highest levels of management 



The steering group will also oversee the development of EMIS as a departmental resource as the system goes 
through a planned redesign. ODE expects to hold the first meeting of this steering group in June 1998. 

LOEO Recommendation: Improve technical support and communication. 

1. Shift the EMIS focus from reporting to analysis. 

2. Improve the EMIS software to increase user-friendliness. 

3. Improve the ODE web site to provide EMIS data. 

ODE has undertaken two projects, one short-term (expected to be implemented within 12 months) and one 
longer-term (expected to be implemented within 18 to 24 months) that address these three recommendations. 

In the short-term, ODE is developing a data warehouse to support EMIS reporting and analysis. This project will 
begin with the release of an RFP for design services in July 1998. The data warehouse will take data from 
EMIS and combine it with data from other departmental systems (primarily financial systems including those 
systems that generate data regarding grant activities) creating one comprehensive database especially designed 
for analysis. The data warehouse will be constructed using advanced relational database technology and will 
employ modem decision support software tools. 

Using this architecture, the data warehouse will provide support for all types of queries; from complex statistical 
analysis to everyday reporting requirements. The decision support software will not require the use of 
complicated programming and will provide access to the data for a wide variety of users across the network. In 
particular, the decision support software’s capabilities for analysis will be extended to ODE’s web site, where 
they can be exploited by users seeking access to the department’s data across the Internet. 

In the longer-term, ODE is pursuing development of new EMIS data entry software. As part of H.B. 650, ODE 
was directed to develop a new, comprehensive student administration software package for use by Ohio school 
districts for K-12 administration. The department issued an RFP in early 1998 for this purpose. In April, after 
reviewing the vendor responses, the department canceled the RFP and decided to revise its approach to 
development of the administration software. 

The department is now pursuing this software development project with two important ideas in mind; make the 
software easy for districts to use, and design the software so that both district administration and EMIS 
requirements are met but data is entered only once. When these two design goals are satisfied, districts will no 
longer need to enter their data for administrative purposes, and then enter an entirely different set of data to meet 
EMIS requirements. By designing the application correctly, districts will enter operational data and the 
application will generate data elements required for EMIS without the districts needing to perform additional 
data entry activities. The department plans to release a new RFP for this software development effort in July 
1998. 

Coupling these design elements to a relational database architecture, and revising the data acquisition process to 
include greater levels of detailed data (while protecting the confidentiality of individual student information), the 
EMIS application will be converted from a system designed primarily for reporting to a comprehensive data 
source for analysis. 

LOEO Recommendation: Improve the accuracy of EMIS data. 

1. Develop data verification procedures for use by districts. 

As mentioned above in the section titled Improve technical support and communication , by designing the 
Student Administration software application correctly, the collection of EMIS data can be automated and made a 
part of the districts’ entry of administrative data for regular operations. The design of the application will 
include significantly more data validity checking than is possible with the existing EMIS data entry system. As 
cited in the recent report Data Use for Decisionmaking in Ohio School Districts (by Thomas B. Parrish of the 
American Institute for Research), redirecting the power of EMIS to enhance information and data reporting in 
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ways specifically designed to assist local (district) administrators holds the greatest potential for improving the 
productivity and accuracy of the system overall. 

2. Employ fluctuation analysis to check year-to-year changes. 

ODE creates EMIS reports that are sent back to school districts that compare the ADM and numbers of funded 
units with prior year results. Other analysis is performed that compare data elements used in the performance 
accountability standards over a number of years. These are all types of fluctuation analysis efforts. In the short- 
term, ODE will increase its examination of data elements that are collected longitudinally (over a number of 
years). If legislative change occurs to allow for the collection of student-level data, then these multiple year 
reviews may be performed on a student by student basis rather than on an aggregated basis, thus providing for a 
more accurate assessment. 

3. Pursue an EMIS data audit. 

ODE’s division of School Finance currently performs ADM audits that require school districts to compare 
EMIS data to actual school records. ODE proposes to increase the number of data elements reviewed in these 
audits and to include those elements used for funding and for performance accountability standards. ODE will 
review the possibility of working with the State Auditor’s office to develop procedures that districts can follow 
during the annual financial audit to ensure data accuracy, especially for data elements that directly affect 
funding. 

LOEO Recommendation: Improve the accountability of EMIS. 

1. Begin evaluation of data acquisition (DA) sites. 



While it is true that ODE has not begun on-site DA Site evaluations, the department has been developing the 
criteria for those evaluations. That work is now complete and a comprehensive set of criteria are ready for use. 
ODE intends to begin on-site evaluations by using an independent third-party, most likely a management 
consulting firm. An RFP for this purpose is planned for release in July 1998. The department will not use the 
Management Council of the OECN for this purpose due to the obvious conflict of interest issues involved. 

ODE agrees with the LOEO recommendation to withhold DA Site funding from those sites that do not meet 
minimum performance criteria. The department will also examine the need for 24 DA Sites. Where it makes 
business sense to do so, the department will work with the MCOCEN to reduce the number of DA Sites and to 
align funding to strengthen the services of those that remain. 

LOEO Recommendation: Alter the structure and design of EMIS to improve performance. 

1. Redesign the EMIS data entry software and database structure to allow for collection of student-level 
information. 

This recommendation will require action on the part of the Legislature to permit ODE to collect student-level 
information. However, this action need not be seen to be as intrusive as it perhaps has been in previous years. 
By using advanced encryption technology and by employing data coding techniques under the control of school 
districts, student-level information can be provided to EMIS for analysis purposes while fully protecting an 
individual student’s privacy and anonymity. Under this approach, an individual student’s identity will be 
masked from EMIS and will be available only to the district. ODE will provide briefings for Legislators and 
other interested parties to explain this approach and the technology behind it in the near future or on request. 

Assuming that ODE receives direction to acquire student-level information, its approach will be as described 
above under the section Improve technical support and communication. By designing the Student 
Administration application correctly, the collection of EMIS data can be facilitated and burdensome data entry 
requirements for school districts reduced. The design of the application will include the ability to collect 



E R?C 



43 



Page 3 



student-level information, which will already be present due to the need for districts to administer student-level 
functions on a day-to-day basis, by using the application. 

2. Convert the EMIS database architecture to a relational database. 

ODE recognizes the need to convert the EMIS database architecture to a relational database. The current EMIS 
database is archaic and obsolete. It does not readily support the reporting and analysis needs of the department 
or other interested parties. 

In May, ODE committed to purchase relational database software for use in several projects. The department 
purchased software from Oracle Corporation. Oracle software is well supported in government and industry and 
is architecturally in alignment with the State of Ohio Information Architecture. 

In the short-term, this technology will be used to construct the ODE data warehouse. Part of this purchase will 
also be used to “seed” DA Sites with relational database technology so that they can better serve their member 
districts and to provide an architectural foundation for development of ODE’s school administration software 
application. 

In the longer-term, ODE’s revised approach to the school administration application includes a redesign of the 
EMIS database. As described above, the application will by design support both a school district’s 
administrative needs and the department’s EMIS data requirements. The relational database technology 
purchased by ODE will provide for the needed flexibility to keep critical operational data at a school district 
(either at a DA Site or, in the case of large city districts, at a district data center) while providing access to it for 
EMIS purposes. This redesign effort will be a major systems development project for ODE and will require a 
funding investment. ODE will request funds for this purpose in its upcoming biennial budget request. 



cc: Cynthia DeJacimo 
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